Benchmarking CLI¶

CLI benchmarking tools measure the wall-clock execution time of shell commands with statistical rigor, accounting for variance, warmup effects, and caching. They are useful for comparing the performance of alternative implementations, build configurations, or tool versions.

Hyperfine¶

https://github.com/sharkdp/hyperfine

Features:
- Statistical analysis across multiple runs.
- Support for arbitrary shell commands.
- Constant feedback about the benchmark progress and current estimates.
- Warmup runs can be executed before the actual benchmark.
- Cache-clearing commands can be set up before each timing run.
- Statistical outlier detection to detect interference from other programs and caching effects.
- Export results to various formats: CSV, JSON, Markdown, AsciiDoc.
- Parameterized benchmarks (e.g. vary the number of threads).
- Cross-platform

Benchmarking CLI¶

Hyperfine¶

See also¶